A Randomized Controlled Trial on the Impact of Polyglot Programming in a Database Context

Using more than one programming language in the same project is common practice. Often, additional languages might be introduced to projects to solve specific issues. While the practice is common, it is unclear whether it has an impact on developer productivity. In this paper, we present a pilot study investigating what happens when programmers switch between programming languages. The experiment is a repeated measures double-blind randomized controlled trial with 3 groups with various kinds of code switching in a database context. Results provide a rigorous testing methodology that can be replicated by us or others and a theoretical backing for why these effects might exist from the linguistics literature

Variation Factors in the Design and Analysis of Replicated Controlled Experiments - Three (Dis)similar Studies on Inspections versus Unit Testing

Background. In formal experiments on software engineering, the number of factors that may impact an outcome is very high. Some factors are controlled and change by design, while others are are either unforeseen or due to chance. Aims. This paper aims to explore how context factors change in a series of for- mal experiments and to identify implications for experimentation and replication practices to enable learning from experimentation. Method. We analyze three experiments on code inspections and structural unit testing. The first two experiments use the same experimental design and instrumentation (replication), while the third, conducted by different researchers, replaces the programs and adapts defect detection methods accordingly (reproduction). Experimental procedures and location also differ between the experiments. Results. Contrary to expectations, there are significant differences between the original experiment and the replication, as well as compared to the reproduction. Some of the differences are due to factors other than the ones designed to vary between experiments, indicating the sensitivity to context factors in software engineering experimentation. Conclusions. In aggregate, the analysis indicates that reducing the complexity of software engineering experiments should be considered by researchers who want to obtain reliable and repeatable empirical measures

Testing coupling relationships in object-oriented programs

As we move toward developing object‐oriented (OO) programs, the complexity traditionally found in functions and procedures is moving to the connections among components. Different faults occur when components are integrated to form higher‐level structures that aggregate the behavior and state. Consequently, we need to place more effort on testing the connections among components. Although OO technologies provide abstraction mechanisms for building components that can then be integrated to form applications, it also adds new compositional relations that can contain faults. This paper describes techniques for analyzing and testing the polymorphic relationships that occur in OO software. The techniques adapt traditional data flow coverage criteria to consider definitions and uses among state variables of classes, particularly in the presence of inheritance, dynamic binding, and polymorphic overriding of state variables and methods. The application of these techniques can result in an increased ability to find faults and to create an overall higher quality software

Usability of Programming Languages

Programming languages form the interface between programmers (the users) and the computation that they desire the computer to execute. Although studies exist for some aspects of programming language design (such as conditionals), other aspects have received little or no human factors evaluations. Designers thus have little they can rely on if they want to make new languages highly usable, and users cannot easily chose a language based on usability criteria. This SIG will bring together researchers and practitioners interested in increasing the depth and breadth of studies on the usability of programming languages, and ultimately in improving the usability of future languages.nonPeerReviewe

An empirical comparison of program auralization techniques

This thesis presents a new approach to using music for human computer interaction, layered program auralization. I use layers of musical structure to represent the state and behavior of a computer program while it is running, taking advantage of metaphorical relationships between musical structure and programming constructs. Layers overlap one another, and can intelligently collaborate to create meaningful mappings from program state or behavior to music. I describe three possible layers in this new system. A dynamically controlled tonal structure changes the harmony while a computer program is running. Program state is represented by changes in the orchestration during execution. Lyrics add semantic information that is difficult to represent with music alone. One possible application of layered program auralization is in debugging runtime behavior of computer programs. Three programs, with faults strategically added, were written to test the effectiveness of layered program auralization. The three programs created included a roulette game, a bank automatic teller machine, and an address book. An empirical study was conducted comparing the effectiveness of three groups of participants while debugging these programs. The first group of participants were given no auralizations, the second strictly musical auralizations, and the third musical auralizations with additional lyrics. Three sessions of experiments were run, the first of which without training into how the auralizations work. In the last two sessions of the experiment, participants in the music and music plus voice groups were given training in the auralizations. Results indicate that layered program auralization was effective in the music group for the bank example, but may not work for every type of computer program, using the current auralization design. Interestingly, users debugging control flow aspects of programs found more errors than other types of programs, like linked structures. In addition, in the music plus voice group, subjects were found to debug less effectively than the control group if they were given no training, although this effect was not seen with the music only group

Evaluating prosodic cues as a means to disambiguate algebraic expressions: an empirical study

The automatic translation of written mathematical expressions to their spoken equivalent is a difficult task. Written mathematics makes use of specialized symbols and a 2-dimensional layout that is hard to translate into clear and unambiguous spoken words. Our approach is to use prosody to help listeners follow along to mathematical expressions spoken aloud with text-to-speech synthesized voices. To achieve this, we developed and empirically tested XSL transformation rules that automatically translate mathematical expressions marked-up with Presentation MathML into corresponding markup using the Speech Synthesis Markup Language (SSML). In this paper, we report on the results from an empirical study we conducted that showed that the simple insertion of pauses inside spoken mathematical expressions dramatically improved subjects\u27 ability to disambiguate between two similar algebraic expressions. Result from our study should benefit designers of screen readers and related audio-based tools that produce spoken renderings of mathematical expressions

Using Spoken Text to Aid Debugging: An Empirical Study

Comprehending and debugging computer programs are inherently difficult tasks. The current approach to building program execution and debugging environments is to use exclusively visual stimuli. We present an alternative: the Sonified Omniscient Debugger (SOD), a program execution and debugging environment designed to output carefully chosen spoken auditory cues to supplement visual stimuli. Originally designed for the blind, earlier work suggested that SOD may benefit sighted programmers as well. We evaluate the SOD environment in a formal debugging experiment comparing 1) a visual debugger, 2) an auditory debugger, and 3) a multimedia debugger, which includes both the visual and auditory stimuli. Our results indicate that while auditory debuggers on their own are significantly less effective for sighted users when compared with visual and multimedia debuggers, multimedia debuggers might benefit sighted programmers under certain circumstances. Specifically, we found that while multimedia debuggers do not provide instant usability, once programmers have some practice, their performance improves under certain metrics

Empirical studies on programming language stimuli

Comprehending and debugging computer programs are inherently difficult tasks. The current approach to building program execution and debugging environments is to use exclusively visual stimuli on programming languages whose syntax and semantics has often been designed without empirical guidance. We present an alternative: Sodbeans, an open-source integrated development environment designed to output carefully chosen spoken auditory cues to supplement empirically evaluated visual stimuli. Originally designed for the blind, earlier work suggested that Sodbeans may benefit sighted programmers as well. We evaluate Sodbeans in two experiments. First, we report on a formal debugging experiment comparing (1) a visual debugger, (2) an auditory debugger, and (3) a multimedia debugger, which includes both visual and auditory stimuli. The results from this study indicate that while auditory debuggers on their own are significantly less effective for sighted users when compared with visual and multimedia debuggers, multimedia debuggers might benefit sighted programmers under certain circumstances. Specifically, we found that while multimedia debuggers do not provide instant usability, once programmers have some practice, their performance in answering comprehension questions improves. Second, we created and evaluated a pilot survey analyzing individual elements in a custom programming language (called HOP) to garner empirical metrics on their comprehensibility. Results showed that some of the most widely used syntax and semantics choices in commercial programming languages are extraordinarily unintuitive for novices. For example, at an aggregate level, the word for , as in a for loop, was rated reliably worse than repeat by more than 673% by novices. After completing our studies, we implemented the HOP programming language and integrated it into Sodbeans